Deep HMResNet Model for Human Activity-Aware Robotic Systems

Endowing the robotic systems with cognitive capabilities for recognizing daily activities of humans is an important challenge, which requires sophisticated and novel approaches. Most of the proposed approaches explore pattern recognition techniques which are generally based on hand-crafted features or learned features. In this paper, a novel Hierarchal Multichannel Deep Residual Network (HMResNet) model is proposed for robotic systems to recognize daily human activities in the ambient environments. The introduced model is comprised of multilevel fusion layers. The proposed Multichannel 1D Deep Residual Network model is, at the features level, combined with a Bottleneck MLP neural network to automatically extract robust features regardless of the hardware configuration and, at the decision level, is fully connected with an MLP neural network to recognize daily human activities. Empirical experiments on real-world datasets and an online demonstration are used for validating the proposed model. Results demonstrated that the proposed model outperforms the baseline models in daily human activity recognition.Comment: Presented at AI-HRI AAAI-FSS, 2018 (arXiv:1809.06606

Approches hybrides pour la reconnaissance du contexte dans les systèmes d'assistance à l'autonomie à domicile : application à la reconnaissance des émotions et à la reconnaissance et l'anticipation de l'activité humaine

The challenges surrounding the ubiquitous robotics are numerous in terms of fields of applications. One of the most important applications is the home care of the elderly and dependent people in the context of the Silver economy (or senior economy). The integration of service robotics in ambient intelligence environments aims to create cyber-physical spaces that provide, anywhere and anytime, a wide range of services to improve the quality of life, the physical and mental state, and the social well-being of users. The objective is to create a unified ecosystem exploiting all connected objects/entities in the environment (sensors, actuators, smartphones, smart TV, digital tablets, smartwatches, service robots, etc.) to create intelligent services and spaces according to the vision of theWeb of Objects (Web of Things). The success of intelligent ambient robotics operating and collaborating with humans in daily living environments depends on their ability to generalize and learn human movements, and obtain a shared understanding of an observed scene. In this context, human daily activity recognition, human emotion recognition, and human intention anticipation are the most challenging cognitive capabilities that should be integrated with any AAL system to guarantee the people's well-being and safety in the ambient intelligence environments. However, to efficiently integrate those capabilities in a highly dynamic environment, multiple modalities sensing systems combined with complex knowledge representation and fusion techniques are required.The objective of this thesis is to propose novel hybrid approaches that enable the AAL system to detect the emotional states, actions, and intentions of users, taking advantage of their context and the benefits of combining data-driven and knowledge-based techniques.The contributions of this thesis can be summarized as follows:Le paradigme de l'Internet des Robots et des objets (IoRT) étend la portée du concept traditionnel de l'internet des objets en dotant n'importe quel objet des trois principales fonctions typiques de tout système robotique : la perception, l'actionnement et le contrôle. Du domaine de la robotique en nuage (cloud robotics) au domaine des systèmes cognitifs en nuage (Cognitive Cloud) pour l'Internet des Robots et des objets (IoRT), il y a un consensus croissant sur la nécessité d'accroitre les capacités cognitives des objets et des robots connectés produits aujourd'hui. L'ancrage est une capacité cognitive importante que doit posséder tout système IoRT. Ce concept est défini comme le processus de création et de maintien des associations entre les descriptions et les informations perceptuelles correspondant aux mêmes objets physiques. Dans la majorité des tâches, les robots doivent percevoir ou interagir avec des objets physiques de leur environnement ; souvent, ils doivent aussi communiquer et raisonner sur les objets et leurs propriétés. Les informations sur les objets sont généralement produites, représentées et utilisées de différentes manières dans divers sous-systèmes robotiques. En particulier, les sous-systèmes de haut niveau raisonnent souvent sur les propriétés et descriptions des objets, tandis que les sous-systèmes de bas niveau utilisent des représentations basées sur les données capteurs. Contrairement aux humains, dans les systèmes multi-agents, les agents peuvent échanger des descriptions et des informations perceptuelles selon le paradigme «voir le monde à travers les yeux des autres».Cette thèse vise à proposer un cadre générique permettant de mettre en œuvre des mécanismes d'ancrages symboliques des observations des systèmes IoRT dans un contexte dynamique. Il s’agit ici d’aller au-delà des approches actuelles qui se focalisent essentiellement sur l’ancrage statique. Ce cadre sera structuré en deux niveaux :•Au niveau bas, le cadre exploitera les approches orientées données et notamment celles basées sur l’apprentissage automatique afin de mettre en œuvre une couche de fusion des données sensorielles de bas niveau. Ce cadre vise en particulier à mettre en œuvre une méthode de reconnaissance coopérative de l'intention humaine et des activités humaines tenant compte de l’incertitude dans les observations.•Au niveau haut, le cadre représentera l'information dynamique (encrage dynamique) en utilisant des modèles conceptuels, permettant d'associer différents types de descriptions d'objets aux informations perceptuelles hétérogènes.•Des simulations et des expérimentations de cas d’utilisation réels seront mises en œuvre pour la validation du cadre proposé

Towards Semantic Multimodal Emotion Recognition for Enhancing Assistive Services in Ubiquitous Robotics

International audienceIn this paper, the problem of endowing ubiquitous robots withcognitive capabilities for recognizing emotions, sentiments,affects and moods of humans, in their context, is studied. Ahybrid approach based on multilayer perceptron (MLP) neural network and n-ary ontologies for emotion-aware roboticsystems is proposed. In particular, an algorithm based on thehybrid-level fusion, an expressive emotional knowledge representation and reasoning model are introduced to recognizecomplex and non-observable emotional context of the user.Empirical experiments on real-world dataset corroborate itseffectiveness

Deep CNN and Probabilistic DL Reasoning for Contextual Affordances

International audienceEndowing robots with cognitive capabilities for recognising contextual object affordances is a big challenge, which requires sophisticated and novel approaches. In this paper, we propose a hybrid approach to interpret contextualised object affordances from sensor data. The proposed approach combines both Deep CNN networks for object and indoor place recognition with probabilistic DL reasoning for affordance inference. We argue that our hybrid approach can be an interesting alternative in situations where no specific dataset for contextualised affordances exists

Deep CNN and Probabilistic DL Reasoning for Contextual Affordances

International audienceEndowing robots with cognitive capabilities for recognising contextual object affordances is a big challenge, which requires sophisticated and novel approaches. In this paper, we propose a hybrid approach to interpret contextualised object affordances from sensor data. The proposed approach combines both Deep CNN networks for object and indoor place recognition with probabilistic DL reasoning for affordance inference. We argue that our hybrid approach can be an interesting alternative in situations where no specific dataset for contextualised affordances exists

Age estimation from faces using deep learning:a comparative analysis

Abstract Automatic Age Estimation (AAE) has attracted attention due to the wide variety of possible applications. However, it is a challenging task because of the large variation of facial appearance and several other extrinsic and intrinsic factors. Most of the proposed approaches in the literature use hand-crafted features to encode ageing patterns. Deeply learned features extracted by Convolutional Neural Networks (CNNs) algorithms usually perform better than hand-crafted features. The main contribution of this paper is an extensive comparative analysis of several frameworks for real AAE based on deep learning architectures. Different well-known CNN architectures are considered and their performances are compared. MORPH, FG-NET, FACES, PubFig and CASIA-web Face datasets are used in our experiments. The robustness of the best deep estimator is evaluated under noise, expression changes, “crossing” ethnicity and “crossing” gender. The experimental results demonstrate the high performances of the popular CNNs frameworks against the state-of-art methods of automatic age estimation. A Layer-wise transfer learning evaluation is done to study the optimal number of layers to fine-tune on AAE task. An evaluation framework of Knowledge transfer from face recognition task across AAE is performed. We have made our best-performing CNNs models publicly available that would allow one to duplicate the results and for further research on the use of CNNs for AAE from face images

Effects of hospital facilities on patient outcomes after cancer surgery: an international, prospective, observational study

© 2022 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 licenseBackground: Early death after cancer surgery is higher in low-income and middle-income countries (LMICs) compared with in high-income countries, yet the impact of facility characteristics on early postoperative outcomes is unknown. The aim of this study was to examine the association between hospital infrastructure, resource availability, and processes on early outcomes after cancer surgery worldwide. Methods: A multimethods analysis was performed as part of the GlobalSurg 3 study—a multicentre, international, prospective cohort study of patients who had surgery for breast, colorectal, or gastric cancer. The primary outcomes were 30-day mortality and 30-day major complication rates. Potentially beneficial hospital facilities were identified by variable selection to select those associated with 30-day mortality. Adjusted outcomes were determined using generalised estimating equations to account for patient characteristics and country-income group, with population stratification by hospital. Findings: Between April 1, 2018, and April 23, 2019, facility-level data were collected for 9685 patients across 238 hospitals in 66 countries (91 hospitals in 20 high-income countries; 57 hospitals in 19 upper-middle-income countries; and 90 hospitals in 27 low-income to lower-middle-income countries). The availability of five hospital facilities was inversely associated with mortality: ultrasound, CT scanner, critical care unit, opioid analgesia, and oncologist. After adjustment for case-mix and country income group, hospitals with three or fewer of these facilities (62 hospitals, 1294 patients) had higher mortality compared with those with four or five (adjusted odds ratio [OR] 3·85 [95% CI 2·58–5·75]; p<0·0001), with excess mortality predominantly explained by a limited capacity to rescue following the development of major complications (63·0% vs 82·7%; OR 0·35 [0·23–0·53]; p<0·0001). Across LMICs, improvements in hospital facilities would prevent one to three deaths for every 100 patients undergoing surgery for cancer. Interpretation: Hospitals with higher levels of infrastructure and resources have better outcomes after cancer surgery, independent of country income. Without urgent strengthening of hospital infrastructure and resources, the reductions in cancer-associated mortality associated with improved access will not be realised. Funding: National Institute for Health and Care Research

Global variation in postoperative mortality and complications after cancer surgery: a multicentre, prospective cohort study in 82 countries

© 2021 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY-NC-ND 4.0 licenseBackground: 80% of individuals with cancer will require a surgical procedure, yet little comparative data exist on early outcomes in low-income and middle-income countries (LMICs). We compared postoperative outcomes in breast, colorectal, and gastric cancer surgery in hospitals worldwide, focusing on the effect of disease stage and complications on postoperative mortality. Methods: This was a multicentre, international prospective cohort study of consecutive adult patients undergoing surgery for primary breast, colorectal, or gastric cancer requiring a skin incision done under general or neuraxial anaesthesia. The primary outcome was death or major complication within 30 days of surgery. Multilevel logistic regression determined relationships within three-level nested models of patients within hospitals and countries. Hospital-level infrastructure effects were explored with three-way mediation analyses. This study was registered with ClinicalTrials.gov, NCT03471494. Findings: Between April 1, 2018, and Jan 31, 2019, we enrolled 15 958 patients from 428 hospitals in 82 countries (high income 9106 patients, 31 countries; upper-middle income 2721 patients, 23 countries; or lower-middle income 4131 patients, 28 countries). Patients in LMICs presented with more advanced disease compared with patients in high-income countries. 30-day mortality was higher for gastric cancer in low-income or lower-middle-income countries (adjusted odds ratio 3·72, 95% CI 1·70–8·16) and for colorectal cancer in low-income or lower-middle-income countries (4·59, 2·39–8·80) and upper-middle-income countries (2·06, 1·11–3·83). No difference in 30-day mortality was seen in breast cancer. The proportion of patients who died after a major complication was greatest in low-income or lower-middle-income countries (6·15, 3·26–11·59) and upper-middle-income countries (3·89, 2·08–7·29). Postoperative death after complications was partly explained by patient factors (60%) and partly by hospital or country (40%). The absence of consistently available postoperative care facilities was associated with seven to 10 more deaths per 100 major complications in LMICs. Cancer stage alone explained little of the early variation in mortality or postoperative complications. Interpretation: Higher levels of mortality after cancer surgery in LMICs was not fully explained by later presentation of disease. The capacity to rescue patients from surgical complications is a tangible opportunity for meaningful intervention. Early death after cancer surgery might be reduced by policies focusing on strengthening perioperative care systems to detect and intervene in common complications. Funding: National Institute for Health Research Global Health Research Unit